Image processing apparatus, image processing method, and non-transitory computer readable medium

ABSTRACT

An image processing apparatus includes an image acquisition unit, a region specifying unit, and a partial-image recognition unit. The image acquisition unit acquires first image data representing a first image and second image data representing a second image that is an image acquired by performing additional recording on the first image. The region specifying unit specifies a region, where additional recording is performed with respect to the first image, of the second image in accordance with the first image data and the second image data acquired by the image acquisition unit. The partial-image recognition unit recognizes a partial image recorded in the region specified by the region specifying unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 fromJapanese Patent Application No. 2015-248169 filed Dec. 21, 2015.

BACKGROUND

(i) Technical Field

The present invention relates to an image processing apparatus, an imageprocessing method, and a non-transitory computer readable medium.

(ii) Related Art

There are many cases where people fill in and submit printed papersheets (business forms, questionnaires, and the like) by hand. Forexample, people fill in documents or various questionnaires to besubmitted to government and municipal offices and the like. There is ademand that people on the collection side of filled-in paper sheets wanthandwritten characters, and characters and numbers selected using markssuch as a circle in these paper sheets to be automatically read andcollected.

SUMMARY

According to an aspect of the invention, there is provided an imageprocessing apparatus including an image acquisition unit, a regionspecifying unit, and a partial-image recognition unit. The imageacquisition unit acquires first image data representing a first imageand second image data representing a second image that is an imageacquired by performing additional recording on the first image. Theregion specifying unit specifies a region, where additional recording isperformed with respect to the first image, of the second image inaccordance with the first image data and the second image data acquiredby the image acquisition unit. The partial-image recognition unitrecognizes a partial image recorded in the region specified by theregion specifying unit.

BRIEF DESCRIPTION OF THE DRAWINGS

An exemplary embodiment of the present invention will be described indetail based on the following figures, wherein:

FIG. 1 is an external view of an image processing apparatus as anexemplary embodiment of the present invention;

FIG. 2 is a functional block diagram of the image processing apparatus,whose appearance is illustrated in FIG. 1;

FIG. 3 is a diagram illustrating a flowchart showing a process performedby the image processing apparatus illustrated in FIGS. 1 and 2;

FIGS. 4A and 4B are diagrams illustrating an example of a blank documentand an example of a filled-in document;

FIG. 5 is a diagram illustrating an example of a difference image;

FIG. 6 is a diagram illustrating set character recognition regionsoverlying a blank document the same as that illustrated in FIG. 4A; and

FIG. 7 is a diagram illustrating the flowchart of a characterrecognition process illustrated as one step in FIG. 3.

DETAILED DESCRIPTION

In the following, an exemplary embodiment of the present invention willbe described.

FIG. 1 is an external view of an image processing apparatus as anexemplary embodiment of the present invention.

An image processing apparatus 10 illustrated here includes a scanner 20and a notebook personal computer (hereinafter referred to as a “notebookPC”) 30. The scanner 20 and the notebook PC 30 are connected via acommunication cable 40.

The scanner 20 is an apparatus that reads an image recorded on adocument and generates image data. In the case where a document isplaced on a document tray 21 of the scanner 20, when a start button (notillustrated) is pressed or a certain command is issued from the notebookPC 30, the document, which is a one-paper-sheet document, is fed intothe scanner 20. The scanner 20 is provided with a sensor (notillustrated) that is in the scanner 20 and that photoelectrically readsan image on a document. The image recorded on the document which hasbeen fed into the scanner 20 is photoelectrically read, and image datais generated. The document whose recorded image has been read is outputonto a paper output tray 22. In the document tray 21, multiple documentsmay be stacked and placed. The scanner 20 sequentially feeds thereintothe multiple documents on the document tray 21 on apaper-sheet-by-paper-sheet basis, reads images on the documents fedthereinto, and outputs the documents onto the paper output tray 22.

In addition, an upper cover 23 of the scanner 20 may be raised around ahinge arranged on a rear surface side and extending horizontally as thecenter of rotation. A document, which is a one-paper-sheet document, mayalso be read in the case where the upper cover 23 is raised, thedocument is placed under the upper cover 23, and the upper cover 23 isclosed.

Image data acquired through reading performed by the scanner 20 is inputto the notebook PC 30 via the communication cable 40.

The notebook PC 30 is provided with a display screen 31 and a keyboard32, and includes therein devices such as a central processing unit (CPU)and a memory for program execution. In the notebook PC 30, programs areexecuted, and processes are performed in accordance with the executedprograms. The notebook PC 30 according to the present exemplaryembodiment executes an image processing program to be described below.

FIG. 2 is a functional block diagram of the image processing apparatus10, whose appearance is illustrated in FIG. 1.

The image processing apparatus 10 according to the present exemplaryembodiment functionally includes an image reading unit 11, an imageacquisition unit 12, a region specifying unit 13, and a partial-imagerecognition unit 14.

The scanner 20 illustrated in FIG. 1 plays the role of the image readingunit 11. That is, the image reading unit 11 photoelectrically reads animage on a document and generates image data. As described above, thisgenerated image data is input to the notebook PC 30 via thecommunication cable 40 illustrated in FIG. 1.

The image acquisition unit 12, the region specifying unit 13, and thepartial-image recognition unit 14 are functions realized by execution ofthe image processing program in the notebook PC 30. Among these, theimage acquisition unit 12 acquires image data, which has been generatedby the image reading unit 11 and transmitted via the communication cable40, by receiving the image data. Here, this image data includes firstimage data representing a first image and second image data representinga second image that is an image acquired by performing additionalrecording on the first image. The image acquisition unit 12distinguishes the first image data from the second image data inaccordance with the order in which pieces of image data supplied fromthe image reading unit 11 are acquired. Specifically, in the presentexemplary embodiment, the image data transmitted first and representingan image of the first document is recognized as the first image data,and pieces of image data transmitted second and subsequently andrepresenting images of the second and subsequent documents arerecognized as pieces of the second image data.

The region specifying unit 13 specifies regions, where additionalrecording is performed with respect to the first image, of the secondimage in accordance with the first image data and the second image dataacquired by the image acquisition unit 12.

Furthermore, the partial-image recognition unit 14 recognizes partialimages recorded in regions specified by the region specifying unit 13.Specifically, in the present exemplary embodiment, the partial-imagerecognition unit 14 recognizes characters represented by partial imagesrecorded in the regions specified by the region specifying unit 13. Thepartial-image recognition unit 14 treats, as a recognition target, apartial image located on the first image and corresponding to a certainregion specified by the region specifying unit 13. Alternatively, thepartial-image recognition unit 14 treats, as a recognition target, apartial image located on the second image and corresponding to thecertain region specified by the region specifying unit 13, or a partialimage located on the difference image between the first image and thesecond image and corresponding to the certain region specified by theregion specifying unit 13. Regarding the point as to whether the firstimage is to be a recognition target or the second image or thedifference image is to be a recognition target, in the present exemplaryembodiment, a partial image located on the second image or on thedifference image between the first image and the second image andcorresponding to the certain region is treated as a recognition targetin the case where the certain region specified by the region specifyingunit 13 is a blank region on the first image. In the case where thecertain region is not a blank region on the first image, a partial imagelocated on the first image and corresponding to the certain region istreated as a recognition target.

FIG. 3 is a diagram illustrating a flowchart showing a process performedby the image processing apparatus 10 illustrated in FIGS. 1 and 2. Theprocess illustrated in FIG. 3 is started upon reception of a readingcommand issued to the scanner 20 (see FIG. 1).

Here, first, it is determined whether or not there are documents on thedocument tray 21 of the scanner 20 (step S11). In the case where thereis no document on the document tray 21, the process simply ends.

In the case where there are documents on the document tray 21, onedocument stacked on the top of the documents is fed into the scanner 20,an image recorded on the document is photoelectrically read, and imagedata is generated (step S12). The generated image data is transmitted tothe notebook PC 30.

In the notebook PC 30, an image processing program that causes thenotebook PC 30 to execute the following process is executed, and theimage data transmitted from the scanner 20 is received by the notebookPC 30 (step S13).

Next, it is determined whether or not the received image data is theimage data acquired by reading the first document (step S14). In thecase where the captured image data is the image data acquired by readingthe first document, the image data is temporarily stored as a blankdocument (step S15).

Note that, since it is clear that images in a data format are handledhere, image data and images are not specifically distinguished from eachother in terms of expression, and “image data” may also be referred toas “image” or “document”.

In the case where it is determined in step S14 that the received imagedata is the image data acquired by reading a certain document, which isone of the second and subsequent documents, the certain document isrecognized as a filled-in document, and a difference document isgenerated by extracting the difference between the filled-in documentand the blank document temporarily stored in step S15 (step S16).

Here, the blank document and the filled-in document according to thepresent exemplary embodiment correspond to an example of a first image(or first image data) and an example of a second image (or second imagedata), respectively, according to the exemplary embodiment of thepresent invention.

FIGS. 4A and 4B are diagrams illustrating an example of a blank documentand an example of a filled-in document. FIG. 4A shows a blankquestionnaire, that is, a blank document 51. Here, there are threequestions (1) to (3) as the content of the questionnaire, and answers tothe questions (1) and (2) among these three questions are made in a formin which any one of numbers 1 to 5 is selected by being encircled with acircle. The question (3) is a question to which an answer is made in aform in which a questionnaire respondent freely fills in a blank answercolumn 511.

In addition, FIG. 4B shows a filled-in document 52, which is aquestionnaire of the same format as the questionnaire illustrated inFIG. 4A, and in which a questionnaire respondent has written his or heranswers. The number of filled-in documents is not limited to one. Amongmultiple documents sequentially read by the scanner 20, the second andsubsequent documents are treated as filled-in documents on apaper-sheet-by-paper-sheet basis.

In one filled-in document illustrated in FIG. 4B, regarding the question(1), the number “4” is encircled with a circle 521. In addition,regarding the question (2), the number “1” is encircled with a circle522. Furthermore, regarding the question (3), a character string 523written in an answer column, which used to be blank, by a questionnairerespondent is arranged.

FIG. 5 is a diagram illustrating an example of a difference image.

As described above, in step S16 of FIG. 3, a difference image 53 betweenthe blank document temporarily stored in step S15 and one filled-indocument acquired this time among the filled-in documents, which are thesecond and subsequent documents, is generated. In the case where thedifference image 53 between the blank document 51 of FIG. 4A and thefilled-in document 52 of FIG. 4B is generated, only the two circles 521and 522 and the character string 523 written by the questionnairerespondent are extracted.

The description continues with reference to FIG. 3 again.

When the difference image 53 (see FIG. 5) is generated in step S16 ofFIG. 3, the pattern written by the questionnaire respondent appears onthe difference image 53. Next, character recognition regions are set inaccordance with the pattern on the difference image 53 (step S17). Acharacter recognition process is executed on partial images recorded inthe character recognition regions (step S20).

FIG. 6 is a diagram illustrating set character recognition regionsoverlying a blank document the same as that illustrated in FIG. 4A.

Here, three character recognition regions 551, 552, and 553 are set. Inthe character recognition process of step S20, character recognition isperformed on each of the three character recognition regions 551, 552,and 553. Among the three character recognition regions 551, 552, and553, two character recognition regions 551 and 552 correspond to the twocircles 521 and 522 written by the questionnaire respondent (see FIG.4A), and the other character recognition region 553 corresponds to theanswer column 511, which used to be blank before being filled in by thequestionnaire respondent (see FIG. 4A).

Here, a collection of processing for reading image data performed instep S13 and processing for classifying the image data into a blankdocument or a filled-in document in accordance with whether or not theimage data is image data acquired by reading the first document (stepsS14 to S16) corresponds to an example of an image acquisition unitaccording to the exemplary embodiment of the present invention.

In addition, a combination of the difference-image generation processperformed in step S16 and the character-recognition-region extractionprocess performed in step S17 in accordance with the difference imagecorresponds to an example of a region specifying unit according to theexemplary embodiment of the present invention.

Furthermore, the character recognition process of step S20 correspondsto an example of a partial-image recognition unit according to theexemplary embodiment of the present invention.

FIG. 7 is a diagram illustrating the flowchart of the characterrecognition process illustrated as one step in FIG. 3.

Here, first, the difference image 53 is searched for extracted characterrecognition regions from the upper left corner to the lower right corner(step S21). When a character recognition region is found, the foundcharacter recognition region is extracted (step S22).

An example in which three character recognition regions illustrated inFIG. 6 are extracted will be described.

In this case, first, the character recognition region 551, which is theuppermost character recognition region, is extracted. Here, since acharacter recognition region is extracted, it is determined in step S23that there is a character recognition region, and the process proceedsto step S24. In step S24, it is determined whether or not the characterrecognition region 551 is a blank region on the blank document 51 (seeFIGS. 4A and 6).

As is clear from FIG. 6, the number “4” is written in the characterrecognition region 551. Accordingly, although character recognition hasnot yet been performed at this stage, it is determined that thecharacter recognition region 551 is not blank. In this case, the processproceeds to step S26, and character recognition is performed on thenumber “4” described there on the blank document 51 (FIGS. 4A and 6).

Thereafter, the process returns to step S21. The search for extractedcharacter recognition regions is started again from the position next tothe character recognition region 551 extracted this time, and the nextcharacter recognition region 552 is extracted (step S22). The characterrecognition region 552 is also not blank on the blank document 51. Theprocess proceeds to step S26, and character recognition is performed onthe number “1” on the blank document 51.

Likewise, in step S22, the next character recognition region 553 isextracted. The character recognition region 553 is a blank region on theblank document 51. Accordingly, it is determined in step S24 that thecharacter recognition region 553 is blank, and the process proceeds tostep S25. In step S25, character recognition is executed on a characterstring written in the character recognition region 553 on the differenceimage 53 (see FIG. 5).

Note that, in step S25, character recognition may also be executed inaccordance not with the difference image 53 but with an image of thefilled-in document 52 (see FIG. 4B).

Next, the process returns to step S21, and is started again. It is thendetermined in step S23 that there is no more character recognitionregion to be extracted, and the process proceeds to step S27. In stepS27, a recognition-result output process is performed and the series ofpieces of processing illustrated in FIG. 7 ends. In therecognition-result output process, a character-recognition result forthe filled-in document, which is a one-paper-sheet document, treated asa character-recognition target this time is, for example, displayed onthe display screen of the notebook PC 30 or written into a file forrecording of character-recognition results.

Thereafter, as illustrated in FIG. 3, the process is performed on thenext document. When all the documents are processed, the series ofpieces of processing illustrated in FIG. 3 also ends.

In this manner, according to the present exemplary embodiment, answersof questionnaire respondents may be recognized without preset settingssuch that, for positions of marks of a mark sense card, informationregarding what marks at respective positions represent is input inadvance.

Here, in the case of the present exemplary embodiment, rules are setsuch that, among multiple documents sequentially read by the scanner 20,the first document is treated as a blank document and the second andsubsequent documents are treated as filled-in documents. However, in thepresent exemplary embodiment of the present invention, the blankdocument does not always have to be arranged at a specific position suchas at the first paper sheet of documents. The blank document may be, forexample, sandwiched between certain filled-in documents in stackedmultiple filled-in documents. In that case, a process for finding ablank document from among multiple documents is performed by the imageacquisition unit. As an example of the process for finding a blankdocument, processing for sequentially extracting the differences betweenthe first document and the second and subsequent documents is performed,and a certain document for which a difference occurs only for the firstdocument is treated as a blank document. In addition, an image isgenerated by extracting portions common to multiple documents on thedata acquired by reading the multiple documents regardless of whethereach of the multiple documents is a blank document or a filled-indocument, pattern matching is performed between the generated image andeach of the read multiple documents, and the document having the highestdegree of matching may be treated as a blank document.

Alternatively, even when reading of filled-in documents is completed byperforming reading on multiple occasions, reading of a blank document isperformed only once for the same type of document. A blank document readonce is stored, and a blank document corresponding to filled-indocuments read this time may be specified by performing pattern matchingor characteristic-point extraction using the filled-in documents readthis time, or by performing form recognition in which the degree ofmatching of a region encircled by straight lines is used.

Furthermore, in the exemplary embodiment of the present invention,reading of a blank document is not always required. A blank document maybe generated on data, for example, by extracting portions common tomultiple filled-in documents from the multiple filled-in documents. Inthis case, the blank document generated on data by extracting the commonportions corresponds to the first image data representing the firstimage.

In addition, here, the image processing apparatus 10 made from acombination of the scanner 20 and the notebook PC 30 illustrated in FIG.1 has been described; however, an image processing apparatus accordingto the exemplary embodiment of the present invention does not alwayshave to have this form. For example, the functions of an imageprocessing apparatus according to the exemplary embodiment of thepresent invention may also be incorporated into a copier into which ascanner and a printer are integrated or a multifunction machine havingmore functions. Furthermore, the functions of an image processingapparatus according to the exemplary embodiment of the present inventionmay also be incorporated into portable terminals equipped with a camerafunction. In that case, the image reading unit may capture images ofdocuments using the camera function.

The foregoing description of the exemplary embodiment of the presentinvention has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Theembodiment was chosen and described in order to best explain theprinciples of the invention and its practical applications, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with the various modifications as are suited tothe particular use contemplated. It is intended that the scope of theinvention be defined by the following claims and their equivalents.

What is claimed is:
 1. An image processing apparatus comprising: animage acquisition unit that acquires first image data representing afirst image and second image data representing a second image that is animage acquired by performing additional recording on the first image; aregion specifying unit that specifies a region, where additionalrecording is performed with respect to the first image, of the secondimage in accordance with the first image data and the second image dataacquired by the image acquisition unit; and a partial-image recognitionunit that recognizes a partial image recorded in the region specified bythe region specifying unit.
 2. The image processing apparatus accordingto claim 1 further comprising a difference extraction unit that extractsthe difference between the first image and the second image, wherein theregion specifying unit specifies the region, where additional recordingis performed, using the difference extraction unit.
 3. The imageprocessing apparatus according to claim 1, wherein the image acquisitionunit distinguishes the first image data from the second image data inaccordance with the order in which pieces of image data are acquired. 4.The image processing apparatus according to claim 2, wherein the imageacquisition unit distinguishes the first image data from the secondimage data in accordance with the order in which pieces of image dataare acquired.
 5. The image processing apparatus according to claim 1,wherein the partial-image recognition unit recognizes a characterrepresented by the partial image recorded in the region specified by theregion specifying unit.
 6. The image processing apparatus according toclaim 2, wherein the partial-image recognition unit recognizes acharacter represented by the partial image recorded in the regionspecified by the region specifying unit.
 7. The image processingapparatus according to claim 3, wherein the partial-image recognitionunit recognizes a character represented by the partial image recorded inthe region specified by the region specifying unit.
 8. The imageprocessing apparatus according to claim 1, wherein the partial-imagerecognition unit treats as a recognition target a partial image locatedon the first image and corresponding to the region specified by theregion specifying unit.
 9. The image processing apparatus according toclaim 2, wherein the partial-image recognition unit treats as arecognition target a partial image located on the first image andcorresponding to the region specified by the region specifying unit. 10.The image processing apparatus according to claim 3, wherein thepartial-image recognition unit treats as a recognition target a partialimage located on the first image and corresponding to the regionspecified by the region specifying unit.
 11. The image processingapparatus according to claim 1, wherein the partial-image recognitionunit treats as a recognition target a partial image located on the firstimage and corresponding to the region specified by the region specifyingunit, or a partial image located on the second image and correspondingto the region specified by the region specifying unit.
 12. The imageprocessing apparatus according to claim 2, wherein in a case where theregion specified by the region specifying unit is a blank region on thefirst image, the partial-image recognition unit treats as a recognitiontarget a partial image located on the second image and corresponding tothe region or a partial image located on a difference image between thefirst image and the second image and corresponding to the region, and ina case where the region is not a blank region on the first image, thepartial-image recognition unit treats as a recognition target a partialimage located on the first image and corresponding to the region. 13.The image processing apparatus according to claim 4, wherein in a casewhere the region specified by the region specifying unit is a blankregion on the first image, the partial-image recognition unit treats asa recognition target a partial image located on the second image andcorresponding to the region or a partial image located on a differenceimage between the first image and the second image and corresponding tothe region, and in a case where the region is not a blank region on thefirst image, the partial-image recognition unit treats as a recognitiontarget a partial image located on the first image and corresponding tothe region.
 14. An image processing method comprising: acquiring firstimage data representing a first image and second image data representinga second image that is an image acquired by performing additionalrecording on the first image; specifying a region, where additionalrecording is performed with respect to the first image, of the secondimage in accordance with the acquired first image data and the acquiredsecond image data; and recognizing a partial image recorded in thespecified region.
 15. A non-transitory computer readable medium storinga program causing a computer to execute a process, the processcomprising: acquiring first image data representing a first image andsecond image data representing a second image that is an image acquiredby performing additional recording on the first image; specifying aregion, where additional recording is performed with respect to thefirst image, of the second image in accordance with the acquired firstimage data and the acquired second image data; and recognizing a partialimage recorded in the specified region.